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IMAGING METHOD FOR A CAPSULE -TYPE ENDOSCOPE UNIT 

[0001] The present application hereby claims priority 
under 35 U.S.C. §119 on German patent application 
number DE 103 18 205.5 filed April 22, 2003, the 
entire contents of which are hereby incorporated herein 
by reference . 

Field of the Invention 

[0002] The invention generally relates to a computer- 
based 3D imaging method for a wireless endoscope unit. 
Preferbly, it relates to one equipped with a video 
camera, of the size of a capsule that can be swallowed 
by the patient. It further generally relates to a 
medical apparatus for the three-dimensional 
representation v of body cavities of the patient which 
can be used, for example, in the field of noninvasive 
gastroenterological endoscopy . 

Background of the Invention 

[0003] Capsule endoscopy is a new method for diagnosing 
diseases of the gastrointestinal tract, in particular 
in the upper sections of the small intestine (jejunum) . 
This permits a patient-friendly, painless investigation 
of the entire region of the small intestine without 
physical exposure to radiation. This method of 
investigation has the advantage that it is possible to 
inspect areas in which conventional radiological and 
endoscope methods achieve only inadequate diagnostic 
results . 

[0004] In this case, the patient swallows a capsule that 
is equipped with a miniature color video camera, 
supplies endoscopic images from the small intestine, 
and permits painless non-invasive diagnostics. It is 
also possible to conceive inserting the capsule in the 

1 


New Patent Application 
Docket No. 32860-000728/US 

rectum if the aim is solely to carry out a coloscopic 
examination of the large intestine including the 
terminal ileum. 

[0005] The examination begins in this case after fasting 
and can be carried out on an outpatient basis. After 
activation and ingestion of the video capsule , the 
patient can carry on with his customary daily 
activities without a problem and without noticing the 
approximately eight-hour passage of the capsule through 
the gastrointestinal tract. After the capsule has spent 
on average approximately 80 minutes in the stomach, it 
requires approximately 90 minutes on average to pass 
through the small intestine. 

[0006] Such a capsule endoscope and a diagnostic system 
for visualizing the entire large intestine mucosa is 
produced, for exmpel , by the Israeli company Given 
Imaging Ltd. and marketed under the brand n M2A® Imaging 
Capsule" . The M2A® imaging capsule includes a miniature 
color video camera, a light source, a miniature 
transmitter and an antenna. The casing of the capsule 
is produced from a sealed biocompatible special 
material that is resistant to digestive secretions in 
the gastrointestinal tract. 

[0007] The M2A® imaging capsule is swallowed by the 
patient and conveyed through the digestive tract 
because of the peristaltic movement of the 
gastrointestinal musculature. The size of the video 
capsule is 11 x 26 mm, its field of view is about 140°, 
and it weighs about four grams. It can be used to find 
lesions with a size of less than 0.1 mm. During a 
normal (eight-hour) examination process , the capsule 
produces roughly 57,000 images at a rate of two images 
per second. The capsule is evacuated in a natural way 
after having passed through the digestive tract. 
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[0008] During its passage through the small intestine, 
the color video camera takes image sequences that are 
transmitted in the form of ultrashort waves to a 
wireless reception unit that is located outside the 
body, and which the patient carries on a belt around 
his hips, and are stored in a data recorder after being 
demodulated and subjected to low-pass filtering and 
analog- to-digital conversion. The belt, which is 
comfortable to wear, with the reception unit enables 
the patient largely to carry on with his customary 
daily activities during the gastrointestinal 
examination . 

[0009] In addition to the image signal, the capsule can 
also output a locating signal referring to its current 
position. The point is that it includes metal parts 
that are located by eight metal detectors applied to 
the patient's abdominal skin. 

[0010] It is thereby possible to assign the images to the 
respective intestinal section. A computer workstation 
at which the RAPID™ ("Reporting and Processing of 
Images and Data") software developed by Given Imaging 
is installed processes the data and compiles a video 
film of the small intestine as well as relevant 
additional information relating to the digestive tract. 
The doctor responsible has the option of viewing this 
video film in real time, of tracking the position of 
the M2A® imaging capsule during its passage through the 
gastrointestinal tract, and of inspecting individual 
images exactly in freeze mode and processing and 
archiving them. The gastroenterologist can then locate 
any pathologies of the small intestine with the aid of 
the visual information obtained in this way. 
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[0011] In addition to the application of capsule 
endoscopy in the region of the gastrointestinal tract, 
numerous further possible applications are currently- 
being planned. This relates generally to the endoscopic 
examination of cavities in the interior of the body in 
which the movement of the video capsule is not 
prevented by the presence of connective tissue. This 
includes, for example, the endovascular examination of 
the cerebral blood vessels, the endoscopic examination 
of the bronchial tract (bronchoscopy) and the minimally 
invasive endoscopic examination of the abdominal cavity 
and the abdominal and pelvic viscera ( laparoscopy ) . 
When the embodiments of the invention are described 
below with reference to the gastrointestinal tract, it 
is to be understood that the embodiments of the 
invention relates in general to the endoscopy of body 
cavities . 

[0012] WO 01/065995 A3 presents a system and an 
associated method for providing images from the 
interior of the body of a patient who is to be 
examined. The system includes an imaging system and an 
RF transmitter of low signal power for transmitting 
video signals of a CMOS video camera to a reception 
system that is located outside the body. The imaging 
system includes at least one CMOS video camera, at 
least one light source for illuminating a site in the 
interior of the body, and an optical lens system for 
focusing the light beams reflected by the site to be 
examined . 

[0013] WO 02/054932 A2 discloses a system and a method 
for wide angle recording of cavities in the internal 
organs of a patient that are to be examined. The system 
includes at least one image sensor, light sources and 
an optical lens system for recording image sequences 
from the interior of the body. The system can be 
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integrated in a medical unit or fitted on the latter, 
which is used for introduction into cavities in the 
interior of the patient's body. This can be, for 
example, an advancing endoscope, an injection needle, 
or else a video capsule that can be swallowed by the 
patient . 

[0014] The device described in WO 02/0953 51 A2 relates to 
a floating, capsule- type image sensor for examining 
liquid- filled cavities of organs in the interior of a 
patient's body that has the specific weight of water or 
some other volume/mass ratio that permits it to float. 
In an exemplary embodiment of this, the image sensor 
includes an imaging system that is installed in a 
floating housing. 

[0015] A diagnostic unit, system and method for providing 
images from cavities of organs in the interior of a 
patient's body, such as the gastrointestinal tract, for 
example, is disclosed in WO 03/010967 Al . The unit is a 
floating capsule that contains an image sensor, a 
microprocessor and a light source for illuminating a 
cavity in the interior of the body. Moreover, the unit 
includes an optical system including a number of 
filters and positive lenses for focusing the light 
reflected by the cavity walls. The image data recorded 
by the image sensor are digitized, compressed and 
transmitted by an integrated RF transmitter to an RF 
receiver that is located outside the body and is 
preferably worn on a belt around the patient's abdomen. 
The image data received are stored in an internal 
memory chip of the RF receiver. In an exemplary 
embodiment of the invention, the reception unit can be 
connected to a workstation that serves to decompress 
and condition the data stream for the purpose of 
visualizing the recorded images. 
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[0016] The disadvantage of the capsule endoscopes 
described above resides in the inaccurate results of 
inspection, since the video capsule cannot be 
controlled and passes too quickly through specific 
regions in the interior of the body, or does not even 
record them at all. Moreover, uninterrupted 
transmission of images from the interior of the body is 
impossible . 

OBJECT OF THE PRESENT INVENTION 

[0017] An embodiment of the present invention includes an 
object of providing a diagnostic imaging unit and a 
method for obtaining and processing images with the aid 
of which it is possible to produce an improved three- 
dimensional overall representation of body cavities. 

[0018] An object may be achieved according to an 
embodiment of the invention. Other exemplary 
embodiments which develop the idea of the invention in 
a particularly advantageous way are also specified. 

SUMMARY OF THE PRESENT INVENTION 

[0019] In accordance with an object, an embodiment of the 
present invention discloses a computer-based medical 3D 
imaging method for a wireless endoscope unit, equipped 
with a video camera, in the form of a capsule, as well 
as a medical apparatus for the three-dimensional 
representation of, for example, body cavities. The 
imaging system components used in this case are 
suitable, for example, for use in the field of non- 
invasive gastroenterological endoscopy for the 
diagnosis of symptoms and/or lesions in the region of 
the human gastrointestinal tract. Moreover, an 
embodiment of the invention also enables a clinical 
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report to be compiled automatically, as well as 
enabling digital archiving of findings and image data. 


BRIEF DESCRIPTION OF THE DRAWINGS 


[0020] Further features, advantages and possible 
applications of the present invention emerge from the 
following description of preferred exemplary 
embodiments of the invention, which are depicted in the 
drawings presented below, and in which: 


figure 1 shows a side view of a medical apparatus for 
recording and evaluating RF signals from a 
capsule- type endoscope unit according to an 
exemplary embodiment of the present 
invention, 


figure 2 shows a plan view of the capsule-type 
endoscope unit used as imaging system 
component and equipped with a miniature video 
camera, according to an exemplary embodiment 
of the present invention, 

figure 3 shows a sectional illustration of a section 
of the human gastrointestinal tract during 
the carrying out of an endoscopic inspection 
with the aid of a capsule-type endoscope 
unit, and 

figure 4 shows a diagram illustrating the computer- 
assisted image processing procedure for 
producing a three-dimensional representation 
of the surfaces of tissue structures starting 
from the image sequence supplied by the 
capsule-type endoscope unit. 
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DETAILED DESCRIPTION OF THE EMBODIMENTS 

[0021] The first step below is to provide more detail on 
the inventive imaging method as depicted in figures 3 
and 4. Subsequently, the functions of the system 
components depicted in figures 1 and 2 are explained. 
The significance of the pictorial elements provided 
with reference symbols in figures 1 to 4 can be 
gathered from the attached list of reference symbols. 

[0022] A first exemplary embodiment of the present 
invention relates to a computer-assisted 3D imaging 
method for a wireless, capsule-type endoscope unit 200, 
equipped with a miniature color video cameral 2 04, for 
use in, for example, the region of the gastrointestinal 
tract of a patient. In this case, the capsule is 
dimensioned in such a way that it can be swallowed by 
the patient. 

[0023] The capsules are of correspondingly smaller 
dimensions when an embodiment of the invention is 
applied in blood vessels. 

[0024] The color spectrum of the camera can also in this 
case cover the infrared and/or ultraviolet spectral 
region(s). During (passive) transport of the capsule 
endoscope 2 00 through the gastrointestinal tract, for 
example, effected by peristaltic movements of the 
esophagus, stomach and intestine, the video camera 2 04 
records a sequence of individual images (SI) and 
transmits (S2) the usually analog image data in the 
form of electromagnetic RF signals to a reception and 
evaluation device that is located outside the body and 
comprises the reception units 103 and 108 and the host 
computer 106. The received individual images of the 
analog video stream transmitted by the capsule-type 
endoscope unit 2 00 are digitized (S3) there at a frame 
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rate corresponding to the real image update or at a 
substantially constant image refresh rate (if they have 
not already been transmitted in digital form) and 
archived (S4) , and can thereafter be displayed 
graphically (S5) . 

[0025] A pattern recognition algorithm (S6) is executed 
in this case for identifying substantially 
corresponding features, that is to say ones which, for 
example, exceed a prescribed threshold value for the 
correspondence, of chronologically successive 

individual images of the recorded image sequence. In 
order to produce a (pseudo-) three-dimensional 
representation of the surfaces of tissue structures to 
be examined, the host computer 106 concatenates 
individual images, which show spatially coherent 
structures, by superimposing common image features in 
the course of an image processing procedure (S7) . 

[0026] The digital image data for the 3D model calculated 
by the host computer 106 in this way, and which can be 
represented in a simplified way substantially by a bent 
tube in the case of the intestine, are then stored in a 
data memory for later visualization. 

[0027] According to an embodiment of the invention, in 
the concatenation of two coherent individual images 
with the numbers m and n, use is made of the path 
difference 

Ax mtn := x n -K m e IR 3 (in mm), 

covered by the capsule- type endoscope unit 200 and 
loaded with a weighting factor, between the 
instantaneous recording positions x m and x n of the unit 
for recording the two individual images . These 
recording positions can be determined in this case 
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either by evaluating X-ray pictures in which the 
capsule-type endoscope unit 200 can be identified, or 
by evaluating the signal transit times T m and T n of the 
wireless image data transmission (S2) from the video 
capsule 2 00 to the reception units 103 and 108, 
respectively . 

[0028] A sectional illustration of a section of the 
region, recorded by the endoscope unit, which shows the 
individual phases of the movement of the capsule-type 
endoscope unit 200, for example through the small 
intestine of a patient while an endoscopic inspection 
is being carried out to produce a continuous image 
sequence, is illustrated in figure 3. 

[0029] Figure 4 depicts the computer-assisted image 
conditioning process for producing a (pseudo) three- 
dimensional representation of the image sequences 
recorded by the capsule endoscope 2 00, which show, for 
example, the surfaces of tissue structures of the 
mucous membranes on the inner walls of the small 
intestine . 

[0030] Together with the image data of each i-th 
individual image (i in this case being a whole number 
greater than or equal to one) of the recorded image 
sequence, according to the invention the, for example, 
Cartesian space coordinates (xi, y±, z±) of the capsule- 
type endoscope unit 2 00 are transmitted, to one of the 
reception units 103 or 108 for each i-th recording, and 
archived in a data recorder in digital form. 

[0031] A further aspect of an embodiment of the present 
invention relates to a further configuration of this 
method for automatically compiling (S10) and archiving 
(Sll) a clinical report. A pattern recognition 
algorithm is carried out for this purpose, and the 
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first step in this is to calculate (S8) the distance 
squares 

d ij : = d 2 (2£ Mi ^ Rj ) 
for 0 < i < T-l, 0 < j < J-l 

between the image parameters, stored in the form of 
N- dimensional feature vectors x M i, of recorded 
individual images with the image parameters, stored in 
the form of N-dimensional reference vectors x R j, of 
images of diseased tissue structures from a reference 
image database. This is performed by calculating the 

of their 


square of the Euclidean length ||42£ij 
difference vectors &x±j' 

\\2 m N 1 


dfj := d 2 (x Mi/ x Rj ) = 


" 2 n = 0 


where Axij := x Mi -x Rj e IR N . 

[0032] Thereafter, the reference vectors xrj of those 
reference images are determined (S9 ) whose distance 
squares d i:? - are a minimum in relation to the respective 

feature vectors x Mi of the individual images to be 
examined. The data of the findings associated with 
these reference vectors x R j are then read out from the 
reference image database. 

[0033] According to a further aspect of an embodiment of 
the present invention, the position and the orientation 
(for example the instantaneous position, the current 
adjusting angle and/or the path) of the capsule-type 
endoscope unit 2 00 are detected in a wireless fashion 
during the examination and inserted into the (pseudo-) 
three-dimensional representation, visualized via a 
display device 107, of the gastrointestinal tract. 


11 


New Patent Application 
Docket No. 32860-000728/US 


[0034] After the conclusion of the gastroenterological 
endoscopy carried out with the aid of the capsule 
endoscope 200, it is possible, by navigating a cursor 
in a control window of the operator interface, 
represented on a display device 107, of a computer 
program, to produce different camera perspectives for 
showing different spatial views, for example of tissue 
structures, in the interior of the gastrointestinal 
tract by influencing at least two input parameters. 
These two input parameters are, for example, the 
magnitude of the advancing movement in the direction of 
movement of the capsule- type endoscope unit 200, and 
the magnitude of the rotary movement about an axis 
pointing in the direction of movement of the capsule 
endoscope 2 00. 

[0035] An embodiment of the invention permits a spatial, 
temporal and personal decoupling of the operating steps 
required for image generation in an endoscopy (for 
example gastroscopy or coloscopy) , which is of great 
importance in practice: a user can view the image 
sequences independently of the location and time of the 
inspection process that led to the recording of the 
images. This has the advantage, inter alia, that 
redundant inspections of the same sections of the 
gastrointestinal tract can be avoided. Difficulties in 
the acquisition of the images are no longer to be noted 
in the three-dimensional image reconstruction. 

[0036] The second exemplary embodiment, depicted in 
figure 2, of the present invention relates to a 
wireless endoscope unit for non-invasive imaging in the 
region of the gastrointestinal tract of a patient, the 
endoscope unit being the size of a capsule 2 01 that can 
be swallowed by the patient. The endoscope unit 
includes an integrated miniature video camera 2 04 for 
recording (SI) a sequence of individual images, and an 
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RF transmitter 205 for the wireless transmission (S2) 
of analog or digital image data in the form of 
ultrashort waves to the reception and evaluation 
device, which is located outside the body and includes 
the reception unit 103 or 108 and the host computer 
106. 

[0037] According to an embodiment of the invention, the 
capsule endoscope 2 00 has a permanent magnet 2 03 that 
is integrated in the capsule and, after the application 
of an external, spatially varying magnetic field B , 
serves the purpose of moving the capsule- type endoscope 
unit 2 00 under active contactless control through the 
patient's gastrointestinal tract to be examined, 
independently of the external magnetic field. 

[0038] Figure 1 depicts a third exemplary embodiment of 
the present invention that relates to a medical 
apparatus 100 for recording and evaluating RF signals 
that are transmitted by the above-described capsule- 
type endoscope unit 2 00, and for the (pseudo-) three- 
dimensional representation of the images recorded by 
the capsule endoscope 200. The apparatus 100 can be 
used with particular advantage in the course of non- 
invasive gastroenterological endoscopy in the region of 
the human gastrointestinal tract. 

[0039] The apparatus includes a reception unit 103 or 
108, located outside the body, for receiving the image 
sequence transmitted by the capsule-type endoscope unit 
2 00, for example in the form of ultrashort waves. The 
apparatus 100 also includes a computation unit 106 for 
decoding the image data transmitted by the capsule-type 
endoscope unit 2 00 and carrying out an image 
conditioning process for producing a three-dimensional 
representation of the surfaces displayed in the images. 
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[0040] A magnet tube 102 having field coils for 
generating a stationary homogeneous magnetic field B 0 , 

as well as one gradient coil each with an associated 
gradient amplifier 109 for the three Cartesian space 
coordinates x, y and z for locally changing the 
magnetic field in the ±x- , ±y- and/or ±z-directions, 
serve the purpose of exerting control in an active 
contactless fashion on a wireless endoscope unit 200, 
equipped with a permanent magnet 203, for example 
through the gastrointestinal tract of a patient. The 
apparatus 100 further includes an arrangement, 
distributed over the upper body of the patient, of 
metal sensors for locating metal parts of the capsule- 
type endoscope unit 2 00, and a measuring sensor with a 
transponder 105 as interface between the sensor 
arrangement 104 and the computation unit 106. A display 
device 107 connected to the computation unit 106 serves 
for visualizing the image data transmitted by the 
capsule- type endoscope unit 2 00 and conditioned by the 
computation unit 106. 

[0041] Indicated according to an embodiment of the 
invention in this case is a three-sided view that is 
generated by the computation unit 106 by calculating 
virtual sectional views along sections parallel to the 
three orthogonal principal planes of the human body - 
"sagittal" (longitudinal section from front to rear), 
"coronal" (longitudinal section from left to right) 
and/or "transverse" or "axial" (cross section through 
the human body) , and is displayed in three different 
control windows 107a-c of the display device 107. 
Moreover, the recorded image sequence can be played 
back in the form of a video film in real time or in 
fast forward mode in a fourth control window 107d of 
the display device 107. 
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[0042] According to a further exemplary embodiment of the 
present invention, the pseudo-three-dimensional 
representation, visualized via the display device 107, 
of the surroundings of the endoscope unit 2 00 can be 
inspected in the course of a virtual fly- through mode 
by varying the viewing perspective with the aid of 
control signals of an input unit (for example a 
computer mouse or a joystick) . 

[0043] Exemplary embodiments being thus described, it 
will be obvious that the same may be varied in many 
ways. Such variations are not to be regarded as a 
departure from the spirit and scope of the present 
invention, and all such modifications as would be 
obvious to one skilled in the art are intended to be 
included within the scope of the following claims . 
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